In this Appendix, we will derive the fixed-point equations for the order parameters presented in the main text, following and generalising the analysis in Ref. [ Saddle-point equations The saddle-point equations are derived straightforwardly from the obtained free energy functionally extremising with respect to all parameters. The zero-regularisation limit of the logistic loss can help us study the separability transition. N 5 + \ 1 p 0, 1 d 5. (66) As a result, given that \ 2( 0, 1 ], the smaller value for which E is finite is U This result has been generalised immediately afterwards by Pesce et al. Ref. [ 59 ] for the Gaussian case, we can obtain the following fixed-point equations, 8 > > > > > >< > > > > > >: E = Mean universality Following Ref. [ In our case, this condition is simpler than in Ref. [ We see that mean-independence in this setting is indeed verified. Numerical experiments Numerical experiments regarding the quadratic loss with ridge regularisation were performed by computing the Moore-Penrose pseudoinverse solution.

Unsupervised learning (UL)-based solvers for combinatorial optimization (CO) train a neural network that generates a soft solution by directly optimizing the CO objective using a continuous relaxation strategy. These solvers offer several advantages over traditional methods and other learning-based methods, particularly for large-scale CO problems.

This appendix contains additional material for the paper "Optimistic tree search strategies forblack-box combinatorial optimization".

There exists an online convex optimization problem where Adam (and RMSprop) has non-zero average regret, and one of the problem is in the form ft(x)= ( Px, if t mod P =1 x, Otherwise x [ 1,1], P N,P 3 (1) Proof. See [1] Thm.1 for proof. For the problem defined above, there's a threshold of β2 above which RMSprop converge. For the problem defined by Eq. (1), ACProp algorithm converges β1,β2 (0,1), P N,P 3. Proof. We analyze the limit behavior of ACProp algorithm.

The next lemma was used in the proof of Theorem 1.